Examiner characteristics and interrater reliability in a communication OSCE.
نویسندگان
چکیده
OBJECTIVE To identify inter-individual examiner factors associated with interrater reliability in a summative communication OSCE in the 4th study year. METHODS The OSCE consists of 4 stations assessed with a 4-item 5-point global rating instrument. A bivariate secondary analysis of interrater reliability in relation to 4 examiner factors (gender, profession, OSCE experience, examiner training) was conducted. Intraclass correlation coefficients (ICC) were calculated and compared between examiner dyads of different similarity. RESULTS 169 pairwise ratings from 19 different examiners in 16 dyads were analysed. Interrater reliability is significantly higher in examiner dyads of same vs. different gender (ICC=0.76 (95%CI=0.65-0.83) vs. ICC=0.41 (95%CI=0.21-0.57)), in dyads of two clinicians vs. non-clinical/mixed professions (ICC=0.72 (95%CI=0.56-0.83) vs. ICC=0.57 (95%CI=0.41-0.69)), and in dyads with high vs. low/mixed OSCE experience (ICC=0.73 (95%CI 0.50-0.87) vs. ICC=0.56 (95%CI=0.41-0.69)). Participation in recent examiner training had no influence on ICCs. CONCLUSION Better concordance of ratings between clinically active examiners might be a hint for context specificity of good communication. Higher interrater reliability between examiners with same gender may indicate gender-specific communication concepts. PRACTICE IMPLICATIONS Medical faculties introducing summative assessment of communication competence should focus the influence of examiner characteristics on interrater reliability.
منابع مشابه
Examiner effect on the objective structured clinical exam – a study at five medical schools
BACKGROUND The Objective Structured Clinical Examination (OSCE) is increasingly used at medical schools to assess practical competencies. To compare the outcomes of students at different medical schools, we introduced standardized OSCE stations with identical checklists. METHODS We investigated examiner bias at standardized OSCE stations for knee- and shoulder-joint examinations, which were i...
متن کاملValidity and Reliability of pre-internship Objective Structured Clinical Examination in Shiraz Medical School
Introduction: Objective Structured Clinical Examination (OSCE) is one of the most appropriate methods for assessment of clinical skills.Validity and reliability assurance is a mandatory factor for any assessment tool. In Shiraz University of Medical Sciences, medical students’ clinical competences are evaluated by a pre-internship OSCE. This study is designed to examine the validity and reliabi...
متن کاملDevelopment of computer-based OSCE re-examination system for minimizing inter-examiner discrepancy.
The validity of objective structured clinical examination (OSCE) in evaluating clinical skills has been confirmed in a number of studies. However, inter-examiner reliability has been noted as a concern by many educators. To minimize discrepancy between examiners, in application of the OSCE, we have developed an OSCE attitude and skill test re-examination system. Student performance was digitall...
متن کاملDoes Changing Examiner Stations During UK Postgraduate Surgery Objective Structured Clinical Examinations Influence Examination Reliability and Candidates' Scores?
OBJECTIVE Objective structured clinical examinations (OSCE) are widely used for summative assessment in surgery. Despite standardizing these as much as possible, variation, including examiner scoring, can occur which may affect reliability. In study of a high-stakes UK postgraduate surgical OSCE, we investigated whether examiners changing stations once during a long examining day affected marki...
متن کاملReliability and validity testing of an evidence-based medicine OSCE station.
The six competencies of the Accreditation Council for Graduate Medical Education include the lifelong learning skills of evidence-based medicine (EBM)/information mastery. We developed and tested an Objective Structured Clinical Examination (OSCE) station that would measure these skills in family medicine residents. This EBM OSCE station is a 30-minute station within a regular OSCE exam. It use...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Patient education and counseling
دوره 100 6 شماره
صفحات -
تاریخ انتشار 2017